Towards 3 D Hand Tracking using a

In this paper we rst describe how we have constructed a 3D deformable Point Distribution Model of the human hand, capturing training data semi-automatically from volume images via a physically-based model. We then show how we have attempted to use this model in tracking an unmarked hand moving with 6 degrees of freedom (plus deformation) in real time using a single video camera. In the course of this we show how to improve on a weighted least-squares pose parameter approximation at little computational cost. We note the successes and shortcomings of our system and discuss how it might be improved. 1 Motivations There has long been a need for a vision-based hand tracking system which is capable of tracking movement with 6 degrees of freedom (DOF), along with articulation information, whilst being as unintrusive as possible. The use of hand markings or coloured gloves and the need for highly constrained environments are undesirable. Such a system should also be as widely available as possible; this implies low-cost technology, so ideally a single camera input should be used and real-time performance should be possible on a standard workstation. From the plethora of work on vision-based hand tracking, relatively few have tackled the task of extracting full 6 DOF hand position and articulation. Notable successes have been due to Dorner 1], whose goal was American Sign Language interpretation, and Rehg and Kanade 2] who developed a system called DigitEyes. Both made use of a manually-constructed articulated hand model with a priori knowledge of inter-joint distances and valid pivot angles. Dorner relied on multicoloured gloves to aid tracking; Rehg however tracked only from edge information, but had to revert to stereo input in order to track full hand articulation. Neither could achieve real-time performance without the use of specialised hardware, and a homogeneous background was always used. With a view to overcoming these limitations, our tracker is based on a 3D version of the Point Distribution Model (PDM) 3]; this is a statistically-derived deformable model which affords several advantages: The model is constructed from real-life examples of hands in various positions, giving an accurate model which implicitly captures the ways in which a hand's shape can and can't vary. The speciicity of the model proves to be invaluable when tracking from a single 2D image, from which data is both noisy and relatively sparse. The hand is modelled as a surface mesh, from …